EM Initialisation for Bernoulli Mixture Learning
نویسندگان
چکیده
Mixture modelling is a hot area in pattern recognition. This paper focuses on the use of Bernoulli mixtures for binary data and, in particular, for binary images. More specifically, six EM initialisation techniques are described and empirically compared on a classification task of handwritten Indian digits. Somehow surprisingly, we have found that a relatively good initialisation for Bernoulli prototypes is to use slightly perturbed versions of the hypercube centre.
منابع مشابه
Learning Mixtures of Bernoulli Templates by Two-Round EM with Performance Guarantee
Dasgupta and Shulman [1] showed that a two-round variant of the EM algorithm can learn mixture of Gaussian distributions with near optimal precision with high probability if the Gaussian distributions are well separated and if the dimension is sufficiently high. In this paper, we generalize their theory to learning mixture of high-dimensional Bernoulli templates. Each template is a binary vecto...
متن کاملTraining GMMs for Speaker Verification
An established approach to training Gaussian Mixture Models (GMMs) for speaker verification is via the expectation-maximisation (EM) algorithm. The EM algorithm has been shown to be sensitive to initialisation and prone to converging on local maxima. In exploration of these issues, three different initialisation methods are implemented, along with a split and merge technique to ‘pull’ the train...
متن کاملStagewise Learning for Sparse Clustering of Discretely-Valued Data
We develop an algorithm to learn Bernoulli Mixture Models based on the principle that some variables are more informative than others. Working from an information-theoretic perspective, we propose both backward and forward schemes for selecting the informative ’active’ variables and using them to guide EM. The result is a stagewise EM algorithm, analogous to stagewise approaches to linear regre...
متن کاملMapping Energy Landscapes of Non-Convex Learning Problems
In many statistical learning problems, the target functions to be optimized are highly non-convex in various model spaces and thus are difficult to analyze. In this paper, we compute Energy Landscape Maps (ELMs) which characterize and visualize an energy function with a tree structure, in which each leaf node represents a local minimum and each non-leaf node represents the barrier between adjac...
متن کاملLearning mixture models – courseware for finite mixture models of multivariate Bernoulli distributions
Teaching of machine learning should aim at the readiness to understand and implement modern machine learning algorithms. Towards this goal, we often have course exercises involving the student to solve a practical machine learning problem involving a reallife data set. The students implement the programs of machine learning methods themselves and gain deep insight on the implementation details ...
متن کامل